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Abstract 


The purpose of this study was to determine variables that predict performance when 
virtual manipulatives are used for mathematics instruction. This study used a quasi- 
experimental design. This design was used to determine variables that predict 
student performance on tests of fraction knowledge for third- and fourth-grade 
students in two treatment groups: classroom instruction using texts and physical 
manipulatives (Cl), and computer lab instruction using virtual fraction applets 
(VM). The Pre-test, Post-test 1, and Post-test 2 measured learning and retention of 
fraction concepts. Observation ethograms documented representation use. The 
results revealed that fewer demographic predictors of student performance (e.g., 
socio-economic status, English language learner status, and gender) exist during 
fraction instruction when virtual manipulatives were used. When instructors used 
virtual manipulatives, there was an equalizing effect on achievement in third and 
fourth grade classrooms, in that fewer demographic factors were influential for VM 
groups compared to Cl groups. 
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Resumen 


El proposito de este arttculo fue como determinar las variables que predicen el 
aprendizaje cuando se utilizan materiales virtuales para ensenar matematicas. En este 
estudio utilizamos una metodologia cuasi-experimental. El diseno se utilizo para 
determinar variables que predicen el aprendizaje de los estudiantes en tests sobre 
conocimiento de fracciones con estudiantes de tercero y cuarto en dos grupos: uno donde 
se utilizaron textos y materiales ffsicos (Cl) y otro donde se usaron applets virtuales 
(VM). El pre-test, post-test 1 y post-test 2 midieron el aprendizaje y retencion de los 
conceptos de fracciones. Se usaron etnogramas para documentar el uso de las 
representaciones. Los resultados rebelan que los predictores demograficos (i.e. estatus 
socio-economico, conocimiento del ingles, genero) tienen poca incidencia cuando se 
usan materiales virtuales. El uso de dichos materiales produce un efecto nivelador entre 
arnbos grupos. 

Palabras clave Materiales virtuales, aprendizaje de fracciones, grados 3 y 4 
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A virtual manipulative (VM) is defined as “an interactive, web-based 
visual representation of a dynamic object that presents 
opportunities for constructing mathematical knowledge,” (Moyer, 
Bolyard & Spikell, 2002, p. 373). In the past two decades, numerous studies 
have shown teachers using virtual manipulatives in a variety of ways (Beck 
& Huse, 2007; Bolyard & Moyer, 2003; Moyer & Bolyard, 2002; Moyer, 
Bolyard & Spikell, 2001; Moyer, Niezgoda, & Stanley, 2005; Moyer- 
Packenham, 2005; Moyer-Packenham, Salkind, Bolyard & Suh, 2013; Suh 
& Moyer, 2007; Suh, Moyer, & Heo, 2005) and research on virtual 
manipulatives has produced mixed results (Baturo, Cooper, & Thompson, 
2003; Clements, Battista & Sarama, 2001; Deliyianni, Michael, & Pitta- 
Pantazi, 2006; Haistings, 2009; Highfield & Mulligan, 2007; Izdorczak, 
2003; Moyer-Packenham & Suh, 2012; Steen, Brooks & Lyon, 2006; 
Takahashi, 2002). A recent meta-analysis comparing the effects of virtual 
manipulatives on student achievement using 32 research reports and 82 
effect size scores reported that virtual manipulatives produced an overall 
moderate effect (0.34) on student achievement when compared with other 
instructional treatments (Moyer-Packenham & Westenskow, 2013). In 
contrast, one of the largest studies comparing virtual manipulatives with 
physical manipulatives and text-based materials in third and fourth grade 
demonstrated no significant differences in achievement between the 
treatments (Moyer-Packenham, Baker, Westenskow, Anderson, Shumway, 
Rodzon, & Jordan, The Virtual Manipulatives Research Group at Utah 
State University, 2013). The results of previous studies demonstrate that the 
relationship between virtual manipulatives and student achievement still 
remains an important area for further study. 

The research reported in this paper is part of a larger research project in 
which researchers examined the use of virtual manipulatives for 
mathematics instruction in third- and fourth-grade classrooms. Multiple 
papers were produced to examine the large amount of data produced in the 
larger study (Anderson-Pence, Moyer-Packenham, Westenskow, Shumway, 
& Jordan, in press; Moyer-Packenham et al., 2013; Westenskow, Moyer- 
Packenham, Anderson-Pence, Shumway, & Jordan, 2014). The first of the 
two larger complementary papers reports on a comparison between physical 
manipulatives and virtual manipulatives and focuses on student 
achievement results in the study (Moyer-Packenham et al., 2013). This 
present paper is the second of the two complementary papers, and here we 
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focus on reporting results about those variables that were predictors of 
student achievement in the larger study. It is not the intention of the authors 
to repeat much of what has already been discussed in the first paper about 
the achievement results, but to build on those results and examine variables 
that predict achievement results when virtual manipulatives are used for 
mathematics instruction. 

Previous Research on Virtual Manipulatives 

Virtual manipulatives are typically considered “cognitive technology tools” 
(Pea, 1985), and as such, are considered a “medium that helps transcend the 
limitations of the mind, such as memory, in activities of thinking, learning, 
and problem solving” (p. 168). As cognitive technology tools, virtual 
manipulatives may change how students approach mathematical tasks. For 
example, a virtual manipulative representing the multiplication of fractions 
(see Fractions - Rectangle Multiplication on http://nlvm.usu.edu) allows 
students to immediately observe the consequences of changing the 
numerator or denominator of either factor (See Figure 1.) By providing 
simultaneous representations in pictorial and symbolic forms, virtual 
manipulatives provide a different type of mathematics experience that 
interacting with physical manipulatives or with text-based materials. 


Multiplication of Fractions 

A|j_ 

▼I ^ 



^ Proper Fractions 
Improper Fractions 



1 x 1=1 =111 

3 3 9 3X3 

i»l =1 =L11 
s i 9 3x3 

A|j_ 

▼I 3 

<* Show Me 
Test Me 


Figure 1. Example of fractions -rectangle multiplication at http://nlvm.usu.edu 
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Among the studies conducted on student achievement and virtual 
manipulatives, there are 12 reported results in favor of virtual 
manipulatives. These studies include: 65 Pre-K to first graders in the 
domain of partitioning (Manches, O'Malley, & Benford, 2010), 68 Pre-K to 
second graders in four mathematics domains (number, geometry, 
measurement, and patterns) (Clements & Sarama, 2007); 31 first graders 
(Steen, Brooks, & Lyon, 2006), 48 third graders (Clements & Battista, 
1989), 560 eighth graders (Pleet, 1991), and 194 tenth graders in the 
domain of geometry (Hauptman, 2010); 32 third graders in the domain of 
measurement (Daghestani, Al-Nuaim & Al-Mshat, 2004), 91 fourth graders 
in the domain of fractions (Ball, 1988), 89 sixth and eighth graders in the 
domain of integers (Smith, 1995), 47 (Cavanaugh, Billan, & Bosnick, 2008) 
and 34 (Guevara, 2009) ninth through twelfth graders in the domain of 
algebra, and 48 university pre-service teachers in the domain of fractions 
(Lin, 2010). Together, these 12 studies represent all of the studies 
conducted to date comparing virtual manipulatives with other instructional 
methods in which there were significant results reported in favor of the 
virtual manipulatives. 

As these studies show, the research on virtual manipulatives has 
included children at different grade levels, different mathematical domains, 
different numbers of participants, and different instructional methods. There 
could be many different variables that contribute to the results obtained in 
these studies. Most of these studies focus on the achievement results when 
different treatments were used with virtual manipulatives. However they do 
not focus on what factors may have predicted those achievement results. 
Therefore the research reported in this paper adds a new dimension to the 
research on virtual manipulatives by investigating what variables might 
predict the outcomes obtained when virtual manipulatives are used for 
mathematics instruction. 


Research Questions 

The study compared two instructional treatments - Treatment 1: classroom 
instruction using physical manipulatives and text-based materials (Cl), and 
Treatment 2: computer lab instruction using virtual manipulatives (VM). 
We examined a number of different variables to determine whether or not 
the variable predicted student achievement in a fraction unit with third- and 
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fourth-grade students. Our research questions in the present study looked 
beyond student achievement and focused on predictors of student 
performance. 

1. Which demographic variables predict third- and fourth-grade student 
achievement, learning, and retention during fraction instruction in two 
treatments (Cl and VM) as indicated by scores on a Pre-test, Post-test 1, 
and Post-test 2 (a delayed post-test)? Within the context of this broad 
research question, we addressed the following sub-question: Does student 
gender, race, objective ability, subjective ability, Socio-Economic Status 
(SES), and English Language Learner (ELL) status predict fraction 
achievement, learning, and retention in either Cl or VM classrooms? 

2. Do mathematics representations, used during instruction or appealing 
on test items, modulate student achievement in Cl and VM classrooms? 
Within the context of this broad research question, we addressed the 
following sub-questions: a) Do students in Cl and VM classrooms score 
differently on symbolic, pictorial, and combined question types? b) Do 
students in Cl and VM classrooms score differently on assessments based 
on their use of different types of representations (e.g., pictorial, symbolic, 
manipulatives) in different participation settings (e.g., individual vs. 
group)? 


Methods 


Participants 

Third graders. A total of 156 third-grade students from eight public 
school classrooms located in four different elementary schools in two 
school districts participated in the study. Third-grade students were 46% 
male/54% female, predominantly Caucasian (75%; 14% Hispanic, 4% 
Mixed Race, 3% African American, 3% Asian, 1% Pacific Islander,), with 
42% of students living in low Socio-Economic Status (SES) households. 
SES households are defined by the school district as those that receive free 
and reduced lunch services. A pre-test identified the group as 14% low-, 
48% average-, and 38% high ability. Only 4% of the third graders were 
identified as English Language Learners (ELL). ELL students are defined 
as those receiving services from the school district to support English 
language development. 
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Fourth graders. A total of 194 fourth-grade students from nine public 
school classrooms located in six different elementary schools in two school 
districts participated in the study. Fourth-grade students were 48% 
male/52% female, predominantly Caucasian (78%; 14% Hispanic, 5% 
Mixed Race, 1% Pacific Islander, 1% Asian), with 54% of students living 
in low SES households. A pre-test identified the group as 21% low-, 49% 
average-, and 30% high ability. There were 8% of fourth graders identified 
as ELL. 

Instructional Treatment Groups 

Researchers used within-class random assignment to assign students to one 
of two treatment groups: Treatment 1: classroom instruction using physical 
manipulatives and text-based materials (Cl), and Treatment 2: computer lab 
instruction using virtual manipulatives (VM). 

Classroom instruction (Cl) treatment groups. The Cl treatment 
groups participated in classroom instruction using physical manipulatives 
and text-based materials. Seven third-grade public school classroom 
teachers taught the eight third-grade Cl classes. Seven fourth-grade public 
school teachers taught the nine fourth-grade Cl classes. Third-grade teacher 
experience ranged from 5 to 23 years (M=17.3); Fourth-grade teacher 
experience ranged from 3 to 32 years (M- 14.6). All teachers (except two) 
taught third or fourth grade for three years or more. The Cl teachers used 
Pearson SuccessNet curriculum materials (Scott Foresman/Addison Wesley 
Mathematics 2005 textbook) to teach the fraction unit. SMART Board™ 
technology was used during mathematics instruction in 50% of third-grade 
and 89% of fourth-grade classrooms. Teachers used manipulatives, 
worksheets, and teacher-created resources during instruction. The teachers 
and classrooms in the Cl group did not use virtual manipulatives during the 
study. 

Virtual manipulatives (VM) treatment groups. The VM treatment 
groups participated in computer lab instruction where teachers introduced 
concepts to students who were working at their own individual computers 
using virtual manipulatives. Four individuals from the local university 
taught the third- and fourth-grade VM groups, including three doctoral- 
level university graduate students and one university faculty member. The 
public school teaching experience of the university-based teachers ranged 
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from 7 to 30 years of experience (M=13.7), which was similar to the 
experience of the Cl teachers. Three of the VM instructors had prior public 
school experience teaching third and fourth grade. The VM teachers also 
used the Pearson SuccessNet curriculum program (Scott Foresman/Addison 
Wesley Mathematics 2005 textbook) and VM instructor-developed task 
sheets specifically designed to teach fraction concepts using the virtual 
manipulatives. These task sheets were designed to mirror the mathematical 
content being taught to the Cl group, with tasks specific to problem 
exploration using the virtual manipulatives. These lesson materials were 
evaluated by an expert group of experienced teachers to determine the 
mathematical content match between Cl and VM lessons. The lesson 
materials were piloted, reviewed and revised in preparation for the research 
project. (See Figure 2 for a sample task sheet.) 



Figure 2. Sample virtual manipulatives task sheet created for the study 

Procedures for Third -and Fourth-Grade Treatment Groups 

At the beginning of the study, each classroom teacher reported 
demographic information including: gender, race, English Language 
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Learner (ELL) status, Socio-Economic Status (SES), and two measures of 
mathematical ability: subjective and objective. Teachers subjectively rated 
students’ mathematical ability as high, medium, or low based on their 
observations of students’ prior performance in mathematics. Objective 
ability was established by comparing each student’s pre-test score to the 
class pre-test average and standard deviation. Standardized scores on the 
pre-test, that were one standard deviation or more above the mean, were 
classified as high, while scores one standard deviation or more below the 
mean were classified as low. All other scores were classified as average. At 
the conclusion of the fraction unit, teachers identified any students absent 
for more than 40% of the fraction unit and we removed these students from 
data analyses. 

For a period of two to three weeks during the study, third- and fourth- 
grade students participated in the study of fractions in the Cl or the VM 
groups. The Cl students learned fraction content in a regular public school 
classroom setting, sitting at their desks, and using manipulatives and text- 
based materials. The Cl teachers introduced the lesson concepts and 
children worked on the mathematics concepts individually or in small 
groups. The VM students also learned fraction content in a regular public 
school setting, in a computer classroom where each child was seated at their 
own computer. The VM teachers introduced the lesson concepts and how to 
use specific virtual manipulatives and children worked independently using 
the virtual manipulatives to complete a variety of mathematical tasks using 
the VM task sheets. 

To ensure instructional fidelity across treatment groups, each paired Cl 
teacher and VM teacher met prior to beginning instruction to specify the 
number of days allotted for the fraction unit and to correlate their lessons 
with the state’s mathematics guidelines. The goal was to ensure that 
students would receive instruction on similar content regardless of their 
treatment groups. Each Cl and VM teacher met daily to discuss plans for 
the following day to further ensure conformity between the two groups. 
During some meetings, teachers determined that students were struggling 
with a particular - concept and together, the instructors decided to re-teach a 
concept. The puipose of this frequent, daily check-in was to ensure that 
students were learning the same mathematical content on each day of the 
study. 
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The length of the fraction units in the eight third-grade classrooms 
ranged from 9 to 12 days (average of 10.1 days) and in the nine fourth- 
grade classrooms ranged from 10 to 17 days (average of 11.4 days). 
Classroom teachers administered all pre- and post-tests to students in both 
treatment groups. All instruction occurred during regularly scheduled 
mathematics classes in the classrooms (Cl groups) and computer labs (VM 
groups) of the participating schools. Students in the VM treatment spent 
almost every day of the fraction unit in the computer lab using their own 
individual computers for approximately 50 minutes each day. 

Third-grade lessons addressed the following concepts: identify the 
denominator of a fraction as the number of equal parts of the unit whole and 
the numerator of a fraction as the number of equal parts being considered; 
define regions and sets of objects as a whole and divide the whole into 
equal parts using a variety of objects, models and illustrations; name and 
write a fraction to represent a portion of a unit whole for halves, thirds, 
fourths, sixths, and eighths; place fractions on the number line and compare 
and order fractions using models, pictures, the number line and symbols; 
find equivalent fractions using concrete and pictorial representations. 
Fourth-grade lessons addressed the following concepts: divide regions, 
lengths, and sets of objects into equal parts using a variety of models and 
illustrations; name and write a fraction to represent a portion of a unit 
whole length or set for halves, thirds, fourths, fifths, sixths, eighths, and 
tenths; generate equivalent fractions and simplify fractions using models, 
pictures, and symbols; order simple fractions; use models to add and 
subtract simple fractions where one single digit denominator is one, two, or 
three times the other; add and subtract simple fractions where one single 
digit denominator is one, two, or three times the other. 

Instruments 

Two primary instruments were used to collect data during the study: 
mathematics content tests and observation ethograms. 

Pre- and post-tests. Third-grade tests contained 13 items total with two 
different types of test items: pictorial items (i.e., pictorial models with a 
written question stem) and combined items (i.e., numerals and operations 
combined with pictorial models with a written question stem). Fourth-grade 
tests contained 19 items total with three different types of test items: 
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pictorial items, combined items, and symbolic items (i.e., numerals and 
operations only). Test questions were selected to: (1) align with the third- 
and fourth-grade objectives for fractions in the state’s mathematics 
curriculum standards; (2) represent a wide range of difficulty levels to 
differentiate among students; and (3) vary the types of representations used 
(i.e., pictorial, symbolic, and combined). During development, 27 multiple- 
choice questions and 3 open-ended questions were compiled into one form 
and checked for content validity by five mathematics educators with 
elementary school experience. Items were piloted with over 500 students 
from 23 classrooms in 14 elementary schools in 6 school districts prior to 
the study. A complete description of the development and validation of 
these instruments is discussed in the complementary paper (Moyer- 
Packenham et ah, 2013). We administered the pre-test immediately prior to 
the fraction unit in each classroom. Post-test 1 on the day after the fraction 
unit concluded, and Post-test 2 seven weeks after the conclusion of the 
fraction unit. 

Observation ethograms. Observation ethograms documented 
instruction in each of the classrooms throughout the study. The puipose for 
these observations was to determine if there were variables that predicted 
achievement that could be observed during instruction. Observers visited 
and observed 70% of the lessons using a modified ethogram protocol. 
Ethograms are instruments most often used by animal behavior researchers 
to efficiently and accurately describe the frequency and duration of 
behaviors made by a species observed in the field without any evaluation of 
the observed behaviors (MacNulty, Mech, & Smith, 2007). The use of this 
instrument results in a cohesive inventory of behavioral patterns that 
describe what a particular species spends its time doing in a particular 
environment. Using an ethogram modified for naturalistic classroom 
observations of humans, observers recorded the types of representations 
used by teachers and students at 5-minute intervals during observations. 
Observers specifically recorded information on mathematical content 
presented, terminology used, mathematical procedures presented, use of 
pictorial models, use of symbolic models, use of physical models, use of 
virtual manipulatives, and students’ access to manipulatives (i.e., passive 
group viewing or active individual manipulation). The four VM teachers 
documented their use of representations in each lesson using instructor logs. 
The logs were subsequently coded and converted to an ethogram protocol. 
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The ethograms provided a quantitative measure of students’ exposure and 
access to various fraction concepts, terminology, and representations in 
each of the treatment groups (Cl and VM). 

Results 

As described previously in this article, the information described in this 
paper is part of a larger study in which student achievement results were 
examined. In the complementary paper to this one, there were no 
statistically significant differences between the VM and Cl groups in terms 
of student achievement on the pre-, post-, and delayed post-tests (Moyer- 
Packenham et al., 2013). To provide a context for the predictor variables 
presented in this present paper, we have included (in Appendix A and B) 
the achievement outcomes for each class that were reported in the Moyer- 
Packenham et al. (2013) paper. In the results that follow, we discuss the 
variables that were predictors of student achievement in our study. 
Essentially these predictors are hidden behind the non-significant statistical 
results when achievement results are presented. 

RQ#1: Post-test 1 and 2, Learning and Retention Analyses 

Our first research question focused on demographic variables that predict 
student achievement, learning, and retention. Data on student demo¬ 
graphics, ability ratings, and descriptions of the instructional environment 
were used as variables in multiple linear regression analyses to determine 
how well each variable predicted student test scores, and which variables 
combined to provide the best fitting predictive model. Using an alpha of .05 
for each analysis, we conducted 24 linear regression analyses at Grade 3 
and 24 at Grade 4 to describe the relationship between the demographic 
variables and students’ test scores (pre-test, Post-test 1, and Post-test 2), 
students’ learning (defined as the difference between Post-test 1 and Pre¬ 
test scores), and students’ retention (defined as the difference between Post¬ 
test 2 and Post-test 1 scores). Table 1 displays a summary of the regression 
results for Grades 3 and 4 based on the mediating variables. 

RQ#1: third-grade analysis. The scores for the VM and Cl groups 
were split to investigate the effects of our predicting variables for third 
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grade. For the Cl group subjective ability, gender, SES, and ELL status, 
were significant predictors of Post-test 1 scores, F( 4, 82)= 9.703, MSE- 
3124.657, p< .000, R 2 - .332, while subjective ability was the only 
significant predictor of Post-test 1 scores for the VM group, F(l,72)= 
6.186, MSE- 1768.818, p- .015, R 2 - .080. The best-fitting model for the 
third-grade Cl group on Post-test 2 included subjective ability and ELL 
status, F(2, 82)= 30.143, MSF= 7574.587, p< .000, F 2 = .43, while the 
model for the VM group consisted only of subjective ability, F (\, 72)= 
30.972, MSE- 8683.746, p< .000, R 2 = .304. Next, we calculated “learning” 
scores for each student, subtracting Pre-test scores from Post-test 1 scores. 
Lor the third-grade Cl group, the model included gender, F(\ , 82)= 4.253, 
MSE= 1773.269, p- .042, R 2 - .050, while the model for the VM group 
included subjective ability, F(l, 72)= 6.362, MSE- 2589.249, p- .014, R 2 = 
.082. 

Table 1 

Linear regression analysis of predictive variables: Test results for grade 3 and 4 



Treatment 

group 

Subjective 

Ability 

Objective 

Ability 

Gender 

SES 

ELL 

Race 

Post-test 1 

CI 

A, □ 


A 

A, □ 

A 



VM 

A 





□ 

Post-test 2 

CI 

A, □ 



□ 

A 



VM 

A 






Learning 

CI 



A 





VM 

A 





□ 

Retention 

CI 


A 

A 





VM 

A 



□ 


□ 


Note. A indicates that the variable was a significant predictor of students’ test 
scores for Grade 3; □ indicates that the variable was a significant predictor of 
students’ test scores for Grade 4. 


These results suggest that students’ gender significantly affected 
performance in the Cl group, while subjective ability affected performance 
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in the VM group. Next we calculated a “retention” score to describe the 
amount of fraction content retained between Post-test 1 and Post-test 2. A 
positive retention score indicates that information was gained between post¬ 
tests. For the third-grade Cl group, the best-fitting model included objective 
ability and gender, F( 2, 82)= 5.305, MSE= 2115.710, p= .007, R 2 = .117, 
while the model for the VM group included only subjective ability, F(\, 
12)- 7.098, MSE- 2614.206, p- .010, R 2 - .091. This indicates that 
objective ability and gender significantly predicts long-term retention of 
fraction concepts in the Cl group. These data show a trend in which a 
broader range of demographic variables influence students’ performance in 
the Cl group compared to the VM group. 

RQ#1 fourth-grade analyses. Next we conducted the same analyses in 
Grade 4 to describe the relationship between demographic variables and 
students’ test scores, learning, and retention. The scores for the VM and Cl 
groups were split to investigate the effects of our predicting valuables for 
fourth grade. For the Cl group the best fitting model for Post-test 1 
consisted of subjective ability and SES, F( 2, 102)= 9.027, MSE- 3210.881, 
p< .000, R 2 - .153), while the model for the VM group included only race, 
F( 1, 90)= 7.865, MSE- 3934.904, /;= .006, R 2 - .081. The preferred model 
for the Cl group alone for predicting Post-test 2 scores included subjective 
ability and SES, F(2, 102)= 14.124, MSF= 3444.874, p< .000, F 2 = .22; 
there were no significant predictors of Post-test 2 for the VM group. For the 
Cl group, there were no significant predictors for learning in Grade 4, while 
the best fitting model for the VM group included race, F (\, 72)= 8.244, 
MSE= 3005.143, p- .005, R 2 = .085. For the Cl group, there were no 
significant predictors for retention in Grade 4, while the best fitting model 
for the VM group included SES and race, F(2, 72)= 9.576, MSE = 2293.532, 
p< .000, R 2 = .179. These results indicate that race and SES significantly 
predict fourth-grade students’ retention scores in the VM but not the Cl 
groups. 

RQ#2: Test Item Question Type Analyses 

The first part of Research Question #2 focused on relationships between 
demographic valuables and test item question types (e.g., pictorial items, 
symbolic items, or combined items). To investigate the effect of our 
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predicting variables on the question types that appeared on the third- and 
fourth-grade tests, we conducted linear regression analyses on one variable 
while controlling for the others. Essentially this examination helped us to 
determine if certain variables predict how well a student will do on different 
types of test items (pictorial items, symbolic items, or combined items). See 
Table 2. 

Table 2 

Linear regression analysis of predictive variables: Question types in grade 3 and 4 



Treatment 

group 

Subjective 

ability 

Objective 

ability 

Gender 

SES 

ELL 

Race 

Post 1: 

CI 

A, □ 



A 

A 


Pictorial 

VM 

A 



□ 



Post 1: 

CI 




□ 



Symbolic 

VM 






□ 

Post 1: 

CI 

A, □ 


A 


A 


Combined 

VM 

□ 




A 

□ 

Post 2: 

CI 

A 



□ 

A 


Pictorial 

VM 

A 






Post 2: 

CI 

□ 



□ 



Symbolic 

VM 







Post 2: 

CI 

A, □ 


□ 

□ 

A 

A 

Combined 

VM 

A 

A 






Note. A indicates that the variable was a significant predictor of students’ test 
scores for Grade 3; □ indicates that the variable was a significant predictor of 
students’ test scores for Grade 4. 


Third-grade analyses. For Post-test 1 pictorial questions alone, the 
best-fitting model for the Cl group consisted of subjective ability, SES, and 
ELL status, F( 3, 82)= 9.897, MSF= 4994.049, p< .000, R 2 = .273, while the 
best-fitting model for the VM group included only subjective ability, 
F(l,72)=4.572, MSE- 1522.539, p- .036, R 2 - .060. For the combined 
question type (e.g., numerals and operations combined with pictorial 
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models with a written question stem), the preferred model for the Cl group 
on Post-test 1 consisted of subjective ability, gender, and ELL status, F(3, 
82)= 8.369, MSE- 3023.058, p< .000, R 2 - .241, while the preferred model 
for the VM group included ELL only, F(1, 72)= 5.483, MSE- 3265.207, p- 
.022, R 2 - .072. Lor the third-grade Cl group, the preferred model for 
pictorial question types on Post-test 2 included subjective ability and ELL 
status, F(2,82)= 23.817, MSF= 7824.735, p< .000, R 2 = .373, while the 
model for the VM group included only subjective ability, F(1, 72)= 29.907, 
MSE- 11787.375, p< .000, R 2 - .296. Lor the Cl group, the best-fitting 
model for the combined question types included subjective ability, ELL 
status, and race, F(3, 82)= 13.589, MSE- 9693.585, p< .000, R 2 - .340, 
while the model for the VM group included subjective and objective ability, 
F(2, 72)= 7.923, MSE- 6048.99, p- .001, R 2 - .185. Consistent throughout 
our analyses, results show fewer significant predicting variables in the VM 
group (compared to the Cl group), suggesting that fewer demographic 
factors mediate students’ performance when fraction concepts are taught 
using virtual manipulatives. 

Fourth-grade analyses. The best fitting model for pictorial question 
types for the Cl group on Post-test 1 included only subjective ability, F(l, 
102)= 12.166, MSE- 13490.445, p= .001, F 2 = .108, while the preferred 
model for the VM group consisted of only SES, F(l, 90)= 10.037, MSE- 
11761.01, p- .002, R 2 = .101. Lor the symbolic question types on Post-test 
1, the preferred model for the Cl group included SES, F(l, 102)= 6.545, 
MSE- 4653.088, p- .012, R 2 - .061, while the preferred model for the VM 
group included race, F(l, 90)= 5.024, MSE- 4401.562, p- .027, R 2 = .053. 
Linally, for the combined question types on Post-test 1, the preferred model 
for the Cl group consisted of only subjective ability, F(l, 102)= 10.141, 
MSE- 4042.156, p- .002, R 2 - .091), while the model for the VM group 
included subjective ability and race, F(2, 90)= 7.469, MSE- 3437.615, p— 
.001, R 2 — .145. Lor the fourth-grade Cl group, the preferred model for 
pictorial question types on Post-test 2 consisted of only SES, F(l, 102)= 
7.284, MSE- 3960.422, p- .008, R 2 - .067. There were no significant 
predictors identified for the VM group. Lor the symbolic question types, the 
preferred model for the Cl group included subjective ability and SES, F(2, 
102)= 9.722, MSE- 6878.904, p< .000, R 2 - .163. There were no significant 
predictors identified for the VM group. Linally, for the combined question 
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types, the preferred model for the Cl group included subjective ability, 
gender, and SES, F( 3, 102)= 6.725, MS£= 2464.694, p< .000, R 2 = .169. 
There were no significant predictors identified for the VM group. Fewer 
significant predicting variables in the VM group compared to the Cl group 
for each test item type suggest that fewer demographic factors mediate 
students’ performance when fraction concepts are taught using virtual 
manipulatives. 

RQ#2: Use of Representations Analyses 

The second paid of Research Question #2 focused on the relationship 
between time students spent using each representation (pictorial, symbolic, 
and physical/virtual manipulatives), type of learning participation 
(individually or in groups), and students’ test scores (Post-test 1, Post-test 2, 
learning, and retention). This examination helped us to understanding how 
using different representations predicted students’ performance on tests and 
how working individually or in groups predicted students’ performance on 
tests. This produced six instructional combinations for analysis: 1) pictorial 
used by an individual student (PI), 2) symbolic used by an individual 
student (SI), 3) manipulatives used by an individual student (MI), 4) 
pictorial used by students in a group (PG), 5) symbolic used by students in 
a group (SG), and 6) manipulatives used by students in a group (MG). 
These abbreviated notations in parentheses are used in the section that 
follows. This enabled comparison of student time spent using each 
instructional representation in groups or working individually, and their 
impact on test scores across treatment groups. We analyzed the ethogram 
observation data by treatment group (Cl and VM) in relation to the 
mathematics test results. These results arc summarized in Table 3. 

Third-grade analyses. A linear regression analysis identified two 
significant predictors of Post-test 1 scores in the Cl group, PI and SI, F( 2,8) 
= 14.58, p = .005, R 2 = .911. Two significant predictors of Post-test 2 
scores were identified in the Cl group, PI and MG, F{ 2,8)= 17.320, p- .003, 
R 2 - .923. One significant predictor of retention scores in the Cl group was 
identified: SI, F(l,8)= 8.199, p- .024, R 2 = .734. No significant predictors 
of learning scores were identified for the Cl group. Furthermore, no 
significant predictors were found for Post-test 1, Post-test 2, learning, or 
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retention scores for the VM group. This suggests that the representations 
used with individual students or with groups of students mediate students’ 
performance in the Cl groups, but not in the VM groups. 

Table 3 

Impact of representations on tests performance by treatment group in grade 3 and 
4 


(H 

H 


Post-test 1 

Cl 


VM 

Post-test 2 

Cl 


VM 

Learning 

Cl 


VM 

Retention 

Cl 


VM 


^3 

3 


’o 

3 

3 


.& -o 


3 

a 

Oh 

o 

o 

a 

c/3 

o 

’> 

£ 

■e 


c/3 

§ *> 

§ 

o 

Oh 


a- 

o 

£ 



a 

c/3 

*3 




o 

£ 

3 

2 

60 

c 

C/3 

3 

2 

60 


A A □ 

Ad □ 


A, □ 


O 


A 


Note. A indicates that the variable was a significant predictor of students’ test 
scores for Grade 3; □ indicates that the variable was a significant predictor of 
students’ test scores for Grade 4. 

Fourth-grade analyses. The fourth-grade analysis identified one 
significant predictor of Post-test 1 scores in the Cl group: PG, F( 1,1 )= 
6.414, p- .045, R 2 = .719. Two significant predictors of Post-test 2 scores in 
the Cl group were identified: SI and SG, F(2,l)~ 10.95, p- .015, R 2 = .902. 
One significant predictor of retention scores in the Cl group was identified: 
SI, F(l,7)= 14.94, p- .008, R 2 - .845. No significant predictors of learning 
scores for the Cl group were identified. Furthermore, no significant 
predictors of Post-test 1, Post-test 2, learning, or retention score for the VM 
group were identified. Similar to Grade 3, no significant predicting 
variables in the VM group (compared to the Cl group) suggest that the 
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representations used with individual students or with groups of students 
mediate students’ performance in the Cl groups, but not in the VM groups. 

Discussion 

The puipose of this study was to examine a number of variables that predict 
achievement results in two instructional treatments: Treatment 1: classroom 
instruction using physical manipulatives and text-based materials (Cl), and 
Treatment 2: computer lab instruction using virtual manipulatives (VM). 

Beyond Student Achievement 

The main overarching finding from this study is that there are less 
extraneous variables (particularly demographic) that predict students’ 
fraction achievement, learning, and retention in third- and fourth-grade 
classrooms when using virtual manipulatives versus text-based materials 
and physical manipulatives. For example, in Grade 3 Cl classrooms, 
subjective ability, gender, SES, and ELL status predicted students’ Post-test 

1 scores, and subjective ability and ELL status predicted students’ Post-test 

2 scores. Similarly in Grade 4, subjective ability and SES predicted Cl 
students’ Post-test 1 scores, and subjective ability and SES predicted Cl 
students’ Post-test 2 scores. Gender predicted overall learning in Grade 3 Cl 
classrooms, and objective ability and gender predicted retention in Grade 3 
Cl classrooms, while there were no significant predictors of learning or 
retention in Grade 4 for the Cl group. In contrast, there were many fewer 
demographic and related predictors of student performance, learning and 
retention in the VM classrooms. In Grade 3, only subjective ability 
predicted VM students’ Post-test 1, Post-test 2, learning, and retention 
scores. In Grade 4, race was the only predictor of students’ Post-test 1, 
learning and retention scores. 

Studies of virtual manipulatives compared with other instructional 
treatments often produce mixed results, or small effects in favor of the 
virtual manipulatives treatments. As this study showed, there is a story that 
goes beyond simply looking at the test results. This is a story about how 
other variables interact while students are learning mathematics. Our results 
shine a spotlight on potential hidden factors that influence student learning 
in mathematics classrooms. Variables that influence mathematics learning 
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may also have the potential to equalized learning when students use virtual 
manipulatives. 

Test Item Question Type Analyses 

For both the Cl and VM groups, subjective ability, which was identified by 
the classroom teacher based on prior student performance, was a frequent 
predictor of student performance for each of the test item question types. 
Other research has shown that students often perform to the level of a 
teacher’s expectations, whether those expectations are high or low for their 
students. The most frequent predictor of student performance on each 
question item type in Cl groups (after subjective ability) was students’ SES, 
followed by ELL status and gender. SES predicted students’ performance 
on Post-tests 1 and 2 in both Grades 3 and 4 in the Cl groups. SES, ELL 
status and gender were rarely predictors in the VM group. The most 
frequent predictor of student performance in VM groups (after subjective 
ability) was race, but this predictor only appeared in Grade 4, and appeared 
less frequently than SES and ELL as predictors in the Cl group. 

Results from the test question type analyses suggest that there are fewer 
predicting variables that mediate overall fraction achievement, learning, and 
retention in third- and fourth-grade classrooms when virtual manipulatives 
are used to teach fraction concepts, rather than text-based materials and 
physical manipulatives. These findings generally hold across test item type, 
whether the question is pictorial, symbolic, or combined. Such findings 
suggest that, regardless of test question type, classroom learning 
opportunities using virtual manipulatives—rather than textbooks and 
physical manipulatives—minimize the impact of extraneous demographic 
variables on learning fractions in third and fourth grade. 

It is important to note, however, that subjective ability, as expected by 
the classroom teacher based on prior student performance, still exerted a 
pervasive influence on student learning in both VM and Cl classrooms. 
Previous studies have demonstrated that this variable strongly impacts 
learning in many contexts (e.g., Rosenthal & Jacobson, 1968). While using 
virtual manipulatives in the classroom may help eliminate the influence of 
some demographic variables—such as SES, ELL status, and gender—on 
student mathematics learning, further research is necessary to determine 
ways in which to reduce other social influences such as subjective teacher 
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expectations. Recent research has begun to address this question in contexts 
other than virtual manipulatives, suggesting that teachers can change 
negative behaviors associated with subjective expectations for students, 
which in turn may positively affect classroom dynamics and student 
learning (e.g., Hamre et al., 2012). 

Students’ Use of Representations 

In terms of representation use, the percentage of time students in third- and 
fourth-grade Cl groups spent using pictorial and symbolic representations 
predicted students’ scores on post-tests. Additionally, the time that third- 
and fourth-grade Cl groups spent working individually with symbolic 
representations predicted student retention of concepts. In contrast, the 
percentage of time third- and fourth-grade VM students spent using 
different representations was not a significant predictor of performance. 

These results suggest that time spent using certain types of instructional 
representations—whether pictorial, symbolic, or physical/virtual 
manipulatives—and the time devoted to individual vs. group work when 
using these representations mediates students’ learning in the Cl groups but 
not in the VM groups. Thus, similar to results discussed above concerning 
the influence of demographic variables on learning in the VM classroom, 
fewer variables relating to time spent using specific representations or 
individual/group contexts predict overall fraction achievement, learning, 
and retention in third- and fourth-grade classrooms when using virtual 
manipulatives versus text-based materials and physical manipulatives. Such 
results are important, as previous studies have shown relationships between 
early mathematics learning and types of representations used for such 
learning with other technology (e.g., Jordan & Baker, 2011; Jordan, 
Suanda, & Brannon, 2008; Moyer-Packenham et al., 2014). Taken together, 
results suggest that student learning with the virtual manipulatives—rather 
than through textbooks and physical manipulatives—helps minimize the 
impact of extraneous demographic and social variables on learning of 
mathematics. 

Conclusion 

In sum, this study determined which demographic variables predict student 
performance in third and fourth grade fraction learning by implementing a 
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large-scale, random-assignment, delayed effects study comparing virtual 
manipulative use with other forms of instruction in the elementary 
classroom. The results revealed that, when this quasi-experimental design 
was used, SES and ELL status more strongly predicted achievement and 
learning in the Cl as compared with the VM classrooms. 

Results in the current study suggest that VM and Cl classrooms provide 
differential opportunities to learn. So what do these results mean for 
children? These results mean that a child from a low-socio economic status 
or who is an English Language Learner in the Cl classrooms already had a 
predetermined outcome on the tests; their ELL and SES status already 
predicted how well they would do on the tests in the Cl classrooms. 
However, in contrast, because there were far fewer predictor variables in 
the VM classrooms, a child’s demographics did not determine what their 
learning outcomes would be. Every child in the VM classrooms began with 
an equal chance. This is a crucial finding. STEM educators must seek to 
minimize the influence of demographic and other extraneous variables on 
student learning and achievement so that all students have access to STEM 
content and STEM-based careers. Such findings highlight the importance of 
further exploring the impact on teaching and learning of multiple 
potentially predictive variables when mathematics students use virtual 
manipulatives in a computer lab setting to learn concepts other than 
fractions, and in grades other than third and fourth. 
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Appendix A. Mean (SD) Third Grade Students Performance: Teacher x 
Treatment Group x Test (Moyer-Packenham et al., 2013) 


Teacher 

Treatment 

group 

Pre-test % 

Post-test 1 % 

Post-test 2 % 

Learning 

Avg. 

Retention 

Avg. 

Mrs. Alpha 

VM 

31.81 

72.72 

43.94 

40.91 

-28.79 



(12.53) 

(19.07) 

(14.56) 

(24.89) 

(23.29) 


PM 

50.90 

85.45 

50.90 

34.55 

-34.55 



(20.93) 

(8.13) 

(26.19) 

(19.71) 

(24.39) 

Mrs. Bravo 

VM 

69.70 

85.86 

72.73 

16.16 

-13.13 



(13.63) 

(8.01) 

(7.87) 

(13.46) 

(10.27) 


PM 

71.71 

79.80 

61.62 

8.10 

-18.20 



(21.53) 

(19.69) 

(20.21) 

(11.53) 

(12.02) 

Mr. Charlie 

VM 

60.61 

75.00 

54.55 

14.40 

-20.45 



(18.32) 

(11.05) 

(15.98) 

(14.22) 

(16.95) 


PM 

48.95 

74.13 

57.34 

25.17 

-16.78 



(22.12) 

(19.58) 

(21.44) 

(22.31) 

(16.94) 

Mrs. Delta* 

VM 

56.06 

59.10 

58.57 

3.03 

-.51 



(19.53) 

(20.26) 

(25.37) 

(25.52) 

(23.01) 


PM 

58.90 

67.99 

56.52 

9.10 

-11.46 



(21.21) 

(17.52) 

(19.56) 

(18.99) 

(21.27) 

Mrs. Echo 

VM 

67.27 

72.72 

69.10 

5.45 

-3.64 



(19.73) 

(18.68) 

(18.77) 

(15.56) 

(13.68) 


PM 

67.27 

75.45 

71.81 

14.54 

-3.63 



(24.26) 

(25.37) 

(18.40) 

(12.27) 

(23.93) 

Mrs. Foxtrot 

VM 

63.64 

69.32 

72.72 

5.68 

3.41 



(17.52) 

(18.78) 

(13.74) 

(10.79) 

(8.32) 


PM 

66.94 

78.51 

72.72 

11.57 

-5.78 



(16.40) 

(13.65) 

(11.49) 

(18.67) 

(11.69) 
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Teacher 

Treatment 

group 

Pre-test % 

Post-test 1 

% 

Post-test 2 

% 

Learning 

Avg. 

Retention 

Avg. 

Mrs. Golf 

VM 

74.54 

70.00 

76.40 

-4.54 

6.36 



(13.41) 

(10.54) 

(13.68) 

(10.71) 

(14.87) 


PM 

58.33 

46.97 

54.54 

-11.36 

7.57 



(21.39) 

(22.21) 

(24.51) 

(15.07) 

(19.31) 

Total 

VM 

60.51 

72.10 

63.99 

11.58 

-8.10 



(16.38) 

(15.20) 

(15.71) 

(16.45) 

(15.77) 


PM 

59.52 

72.61 

60.78 

13.09 

-11.83 



(21.12) 

(18.02) 

(20.26) 

(16.93) 

(18.51) 


Note. The asterisks indicate teachers who taught more than one class. 






REDIMAT ; 3(2) 


149 


Appendix B 

Appendix B. Fourth-Grade Students Performance: Teacher x Treatment 
Group x Test (Moyer-Packenham et al., 2013) 


Teacher 

Treatment 

group 

Pre-test % 

Post-test 1 % 

Post-test 2 % 

Learning 

Avg. 

Retention 

Avg. 

Mr. 

VM 


50.98 

61.06 

42.85 

10.08 

-18.21 

Hotel* 



(18.63) 

(22.51) 

(19.25) 

(16.12) 

(17.34) 


PM 


52.45 

60.78 

40.68 

8.33 

-20.09 




(17.93) 

(19.13) 

(16.63) 

(17.25) 

(16.89) 

Mrs. 

VM 


49.85 

67.50 

54.34 

17.64 

-13.16 

India* 



(20.00) 

(20.93) 

(14.98) 

(20.46) 

(20.45) 


PM 


49.41 

69.11 

46.47 

19.40 

-22.64 




(21.62) 

(19.91) 

(16.84) 

(22.92) 

(16.88) 

Mrs. 

VM 


47.05 

67.37 

45.45 

20.32 

-21.92 

Juliet 



(20.88) 

(18.51) 

(15.07) 

(20.60) 

(13.94) 


PM 


47.89 

60.51 

38.65 

12.60 

-21.84 




(17.39) 

(23.28) 

(16.65) 

(15.27) 

(11.86) 

Mrs. Kilo 

VM 


47.05 

57.98 

45.37 

10.92 

-12.60 




(20.65) 

(19.92) 

(22.19) 

(12.44) 

(20.77) 


PM 


43.53 

58.82 

40.00 

15.29 

-18.82 




(24.04) 

(25.41) 

(17.71) 

(15.23) 

(21.79) 

Mrs. Lima 

VM 


35.94 

67.32 

57.51 

31.37 

-9.80 




(16.49) 

(30.86) 

(22.74) 

(24.07) 

(12.82) 


PM 


39.57 

69.51 

55.08 

29.94 

-14.43 




(14.43) 

(19.65) 

(14.94) 

(14.03) 

(11.87) 

Mrs. Mike 

VM 


61.17 

80.58 

59.41 

19.41 

-21.17 




(18.84) 

(25.27) 

(24.08) 

(17.54) 

(9.86) 


PM 


50.98 

75.49 

64.21 

24.50 

-11.27 




(19.32) 

(20.96) 

(15.35) 

(20.19) 

(13.35) 
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Teacher 

Treatment 

group 

Pre-test % 

Post-test 1 % 

Post-test 2 % 

Learning Avg. 

Retention 

Avg. 

Mrs. 

VM 


42.64 

59.80 

45.09 

17.15 

-14.70 

November 



(16.47) 

(20.80) 

(20.42) 

(23.32) 

(14.94) 


PM 


41.17 

67.64 

51.96 

53.47 

-15.68 




(14.18) 

(19.18) 

(13.23) 

(21.35) 

(17.25) 

Total 

VM 


47.81 

65.94 

50.01 

18.13 

-15.94 




(18.58) 

(23.37) 

(20.04) 

(19.22) 

(16.23) 


PM 


46.43 

65.98 

48.15 

19.55 

-17.83 




(18.42) 

(20.39) 

(15.68) 

(18.03) 

(15.20) 


Note. The asterisks indicate teachers who taught more than one class. 






